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Abstract 

We prove that a random linear code over W q , with probability arbitrarily close to 1, is list 
decodable at radius 1 — 1/ q — e with list size L = 0(l/e 2 ) and rate R = 9 (e 2 /(log 3 (l/e))). Up 
to the polylogarithmic factor in 1 /e and constant factors depending on q, this matches the lower 
bound L = fl q (l/e 2 ) for the list size and upper bound R = O q (e 2 ) for the rate. Previously only 
existence (and not abundance) of such codes was known for the special case q = 2 (Guruswami, 
Hastad, Sudan and Zuckerman, 2002). 

In order to obtain our result, we employ a relaxed version of the well known Johnson bound on 
list decoding that translates the average Hamming distance between codewords to list decoding 
guarantees. We furthermore prove that the desired average-distance guarantees hold for a code 
provided that a natural complex matrix encoding the codewords satisfies the Restricted Isometry 
Property with respect to the Euclidean norm (RIP-2). For the case of random binary linear 
codes, this matrix coincides with a random submatrix of the Hadamard- Walsh transform matrix 
that is well studied in the compressed sensing literature. 

Finally, we improve the analysis of Rudelson and Vershynin (2008) on the number of random 
frequency samples required for exact reconstruction of /c-sparse signals of length N. Specifically, 
we improve the number of samples from 0(k \og(N) log 2 (fc)(log k + log log N)) to (9(fclog(7V) • 
log 3 (fc)). The proof involves bounding the expected supremum of a related Gaussian process by 
using an improved analysis of the metric defined by the process. This improvement is crucial 
for our application in list decoding. 
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1 Introduction 



This work is motivated by the list decodability properties of random linear codes for correcting a 
large fraction of errors, approaching the information-theoretic maximum limit. We prove a near- 
optimal bound on the rate of such codes, by making a connection to and establishing improved 
bounds on the restricted isometry property of random submatrices of Hadamard matrices. 

A q-ary error correcting code C of block length n is a subset of [q] n , where [q] denotes any 
alphabet of size q. The rate of such a code is defined to be (log 9 \C\)/n. A good code C should be 
large (rate bounded away from 0) and have its elements (codewords) well "spread out." The latter 
property is motivated by the task of recovering a codeword c £ C from a noisy version r of it that 
differs from c in a bounded number of coordinates. Since a random string r £ [q] n will differ from 
c on an expected (1 — l/q)n positions, the information-theoretically maximum fraction of errors 
one can correct is bounded by the limit (1 — l/q). In fact, when the fraction of errors exceeds 
5(1 — I/?)) it is not possible to unambiguously identify the close- by codeword to the noisy string 
r (unless the code has very few codewords, i.e., a rate approaching zero). 

In the model of list decoding, however, recovery from a fraction of errors approaching the limit 
(1 — l/q) becomes possible. Under list decoding, the goal is to recover a small list of all codewords 
of C differing from an input string r in at most pn positions, where p is the error fraction (our 
interest in this paper being the case when p is close to 1 — l/q). This requires that C have the 
following sparsity property, called (p, L)-list decodability, for some small L : for every r £ [q] n , there 
are at most L codewords within Hamming distance pn from r. We will refer to the parameter L 
as the "list size" — it refers to the maximum number of codewords that the decoder may output 
when correcting a fraction p of errors. Note that (p, L)-list decodability is a strictly combinatorial 
notion, and does not promise an efficient algorithm to compute the list of close-by codewords. In 
this paper, we only focus on this combinatorial aspect, and study a basic trade-off between between 
p, L, and the rate for the important class of random linear codes, when p — > 1 — l/q. We describe 
the prior results in this direction and state our results next. 

For integers q, L > 2, a random g-ary code of rate R = 1 — h q (p) — 1/L is (p, L)-list decodable 
with high probability. Here h q : [0,1 — l/q] — > [0, 1] is the g-ary entropy function: h q (x) = x log q (q — 
1) — x log q x — (1 — x) log 9 (l — x). This follows by a straightforward application of the probabilistic 
method, based on a union bound over all centers r £ [q] n and all (L + l)-element subsets S 
of codewords that all codewords in S lie in the Hamming ball of radius pn centered at r. For 
p = 1 — l/q — e, where we think of q as fixed and e — > 0, this implies that a random code of rate 
fi g (e 2 ) is (1 — l/q — e, O g (l/e 2 ))-list decodable. (Here and below, the notation Q q and O q hide 
constant factors that depend only on q.) 

Understanding list decodable codes at the extremal radii p = 1 — l/q — e, for small e, is of par- 
ticular significance mainly due to numerous applications that depend on this regime of parameters. 
For example, one can mention hardness amplification of Boolean functions [STV01], construction 
of hardcore predicates from one-way functions [GL89], construction of pseudorandom generators 
[STV01] and randomness extractors [TreOl], inapproximability of NP witnesses [KS99], and approx- 
imating the VC dimension [MU01]. Moreover, linear list-decodable codes are further appealing due 
to their symmetries, succinct description, and efficient encoding. For some applications, linearity 
of list decodable codes is of crucial importance. For example, the black-box reduction from list 
decodable codes to capacity achieving codes for additive noise channels in [GS10], or certain appli- 
cations of Trevisan's extractor [TreOl] (e.g., [ChelO, § 3.6, § 5.2]) rely on linearity of the underlying 
list decodable code. Furthermore, list decoding of linear codes features an interplay between linear 
subspaces and Hamming balls and their intersection properties, which is of significant interest from 
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a combinatorial perspective. 

This work is focused on random linear codes, which are subspaces of F™, where ~F q is the 
finite field with q elements. A random linear code C of rate R is sampled by picking k = Rn 
random vectors in F™ and letting C be their Fg-span. Since the codewords of C are now not 
all independent (in fact they are not even 3-wise independent), the above naive argument only 
proves the (p, L)-list decodability property for codes of rate 1 — h q (p) — l/\og q (L + 1) [ZP82]. 1 
For the setting p = 1 — 1/q — e, this implies a list size bound of exp(O g (l/e 2 )) for random linear 
codes of rate Q, q (e 2 ), which is exponentially worse than for random codes. Understanding if this 
exponential discrepancy between general and linear codes is inherent was raised an open question 
by Elias [Eli91]. Despite much research, the exponential bound was the best known for random 
linear codes (except for the case of q = 2, and even for q = 2 only an existence result was known; 
see the related results section below for more details). 

Our main result in this work closes this gap between random linear and random codes, up to 
polylogarithmic factors in the rate. We state a simplified version of the main theorem (Theorem 13) 
below. 

Theorem 1 (Main, simplified). Let q be a prime power, and let e > be a constant parameter. 
Then for some constant a q > only depending on q and all large enough integers n, a random 
linear code C C F™ of rate a q e 2 / log 3 (l/e) is (1 — 1/q — e, 0(l/e 2 ))-list decodable with probability at 
least 0.99. (one can take a q = 47(1/ log 4 q).) 

We remark that both the rate and list size are close to optimal for list decoding from a (1 — 1 jq— e) 
fraction of errors. For rate, this follows from the fact the g-ary "list decoding capacity" is given by 
1 — h q (p), which is O q {e 2 ) for p = 1 — 1/q — e. For list size, a lower bound of Q q (l/e 2 ) is known 
- this follows from [Bli86] for q = 2, and was shown for all q in [GV10] (and also in [Bli05] under 
a convexity conjecture that was later proved in [Bli08]). We have also assumed that the alphabet 
size q is fixed and have not attempted to obtain the best possible dependence of the constants on 
the alphabet size. 

1.1 Related results 

We now discuss some other previously known results concerning list decodability of random linear 
codes. 

First, it is well known that a random linear code of rate ^ g (e 4 ) is (1 — 1/q — e, 0(l/e 2 ))-list 
decodable with high probability. This follows by combining the Johnson bound for list decoding 
(see, for example, [GS01]) with the fact that such codes lie on the Gilbert- Varshamov bound and 
have relative distance 1 — 1/q — e 2 with high probability. This result gets the correct quadratic 
dependence in list size, but the rate is worse. 

Second, for the case of q = 2, the existence of (p, L)-list decodable binary linear codes of rate 
1 — h{p) — 1/L was proved in [GHSZ02]. For p = 1/2 — e, this implies the existence of binary 
linear codes of rate Q(e 2 ) list decodable with list size 0(l/e 2 ) from an error fraction 1/2 — e. This 
matches the bounds for random codes, and is optimal up to constant factors. However, there are 
two shortcomings with this result: (i) it only works for q = 2 (the proof makes use of this in a 
crucial way, and extensions of the proof to larger q have been elusive) , and (ii) the proof is based 

1 The crux of the argument is that any L non-zero vectors in Fjj must have a subset of log 9 (L + 1) linearly 
independent vectors, and these are mapped independently by a random linear code. This allows one to effectively 
substitute log 9 (L + 1) in the place of L in the argument for fully random codes. 
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on the semi-random method. It only shows the existence of such a code while failing to give any 
sizeable lower bound on the probability that a random linear code has the claimed list decodability 
property. 

Motivated by this state of affairs, in [GHK11], the authors proved that a random q-ary linear 
code of rate 1 — h q (p) — C PA jL is (p, L)-list decodable with high probability, for some C PA < oo that 
depends on p, q. This matches the result for completely random codes up to the leading constant 
C Pi g in front of 1/L. Unfortunately, for p = 1 — 1/q — e, the constant C pq depends exponentially 2 
on 1/e. Thus, this result only implies an exponential list size in 1/e, as opposed to the optimal 
0(l/e 2 ) that we seek. 

Summarizing, for random linear codes to achieve a polynomial in 1/e list size bound for error 
fraction 1 — 1/q — e, the best lower bound on rate was Q(e 4 ). We are able to show that random 
linear codes achieve a list size growing quadratically in 1/e for a rate of f2(e 2 ). One downside of our 
result is that we do not get a probability bound of 1 — o(l), but only 1 — 7 for any desired constant 
7 > (essentially our rate bound degrades by a log(l/7) factor). 

Finally, there are also some results showing limitations on list decodability of random codes. 
It is known that both random codes and random linear codes of rate 1 — h q (p) — rj are, with high 
probability, not (p, c Ptq /r])-\\st decodable [Rudll, GN12]. For arbitrary (not necessarily random) 
codes, the best lower bound on list size is J7(log(l/r/)) [Bli86, GN12]. 

1.2 Proof technique 

The proof of our result uses a different approach from the earlier works on list decodability of 
random linear codes [ZP82, Eli91, GHSZ02, GHK11]. Our approach consists of three steps. 

Step 1: Our starting point is a relaxed version of the Johnson bound for list decoding that only 
requires the average pairwise distance of L codewords to be large (where L is the target list size), 
instead of the minimum distance of the code. 

Technically, this extension is easy and pretty much follows by inspecting the proof of the Johnson 
bound. This has recently been observed for the binary case by Cheraghchi and is implicit in the 
survey [Chell]. Here, we give a proof of the relaxed Johnson bound for a more general setting of 
parameters, and apply it in a setting where the usual Johnson bound is insufficient. Furthermore, 
as a side application, we show how the average version can be used to bound the list decoding 
radius of codes which do not have too many codewords close to any codeword — such a bound 
was shown via a different proof in [GKZ08] , where it was used to establish the list decodability of 
binary Reed-Muller codes up to their distance. 

Step 2: Prove that the L-wise average distance property of random linear codes is implied by the 
order L restricted isometry property (RIP-2) of random submatrices of the Hadamard matrix (or 
in general, matrices related to the Discrete Fourier Transform). 

This part is also easy technically, and our contribution lies in making this connection between 
restricted isometry and list decoding. The restricted isometry property has received much attention 
lately due to its relevance to compressed sensing (cf. [Can08, CRT06a, CRT06b, CT06, Don06]) 
and is also connected to the Johnson-Lindenstrauss dimension reduction lemma [BDDW08]. Our 
work shows another interesting application of this concept. 

2 The constant C Pl9 depends exponentially on l/S p , where q~ Spn is an upper bound on the probability that 
two random vectors in F™ of relative Hamming weight at most p, chosen independently and uniformly among all 
possibilities, sum up (over F") to a vector of Hamming weight at most p. When p — 1 — 1/q — e, we have 8 P = O q (e 2 ) 
which makes the list size exponentially large. 
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Step 3: Prove the needed restricted isometry property of the matrix obtained by sampling rows 
of the Hadamard matrix. 

This is the most technical part of our proof. Let us focus on q = 2 for simplicity, and let H be 
the N x N Hadamard (Discrete Fourier Transform) matrix with N = 2™, whose (x,y)'th entry is 
(_l)<*,v> for x,y e {0,1}™. We prove that (the scaled version of) a random submatrix of H formed 
by sampling a subset of m = 0(fclog 3 /clog N) rows of H satisfies RIP of order k with probability 
0.99. This means that every k columns of this sampled matrix M are nearly orthogonal — formally, 
every m x k submatrix of M has all its k singular values close to 1. 

For random matrices m x N with i.i.d Gaussian or ±1 entries, it is relatively easy to prove 
PJP-2 of order k when m = 0{k\ogN) [BDDW08]. Proving such a bound for submatrices of the 
Discrete Fourier Transform (DFT) matrix (as conjectured in [RV08]) has been an open problem for 
many years. The difficulty is that the entries within a row are no longer independent, and not even 
triple-wise independent. The best proven upper bound on m for this case was 0(Hog 2 k{\ogk + 
log log N) log N) , improving an earlier upper bound 0(klog 6 N) of Candes and Tao [CT06]. We 
improve the bound to 0(k log 3 klogN) — the key gain is that we do not have the log log N factor. 
This is crucial for our list decoding connection, as the rate of the code associated with the matrix 
will be (log N)/m, which would be o(l) if m = f2(log iVloglog N). We will take k = L = 6(l/e 2 ) 
(the target list size), and the rate of the random linear code will be f2(l/(fc log 3 k)), giving the 
bounds claimed in Theorem 1. We remark that any improvement of the RIP bound towards the 
information-theoretic limit m = £l(k\og(N/k)), a challenging open problem, would immediately 
translate into an improvement on the list decoding rate of random linear codes via our reductions. 

Our RIP-2 proof for row-subsampled DFT matrices proceeds along the lines of [RV08], and is 
based on upper bounding the expectation of the supremum of a certain Gaussian process [LT91, 
Chap. 11]. The index set of the Gaussian process is B 2 ' , the set of all fc-sparse unit vectors in H N , 
and the Gaussian random variable G x associated with x € is a Gaussian linear combination of 
the squared projections of x on the rows sampled from the DFT matrix (in the binary case these are 
just squared Fourier coefficients) 3 . The key to analyzing the Gaussian process is an understanding 
of the associated (pseudo)-metric X on the index set, defined by \\x — x'\\x = ~Eq\G x — G x i\ 2 . This 
metric is difficult to work with directly, so we upper bound distances under X in terms of distances 
under a different metric X' . The principal difference in our analysis compared to [RV08] is in the 
choice of X' — instead of the max norm used in [RV08], we use a large finite norm applied to 
the sampled Fourier coefficients. We then estimate the covering numbers for X' and use Dudley's 
theorem to bound the supremum of the Gaussian process. 

It is worth pointing out that, as we prove in this work, for low-rate random linear codes the 
average-distance quantity discussed in Step 1 above is substantially larger than the minimum dis- 
tance of the code. This allows the relaxed version of the Johnson bound attain better bounds than 
what the standard (minimum-distance based) Johnson bound would obtain on list decodability 
of random linear codes. While explicit examples of linear codes surpassing the standard Johnson 
bound are already known in the literature (see [GGR11] and the references therein), a by-product 
of our result is that in fact most linear codes (at least in the low-rate regime) surpass the standard 
Johnson bound. However, an interesting question is to see whether there are codes that are list 
decodable even beyond the relaxed version of the Johnson bound studied in this work. 

3 We should remark that our setup of the Gaussian process is slightly different from [RV08], where the index set 
is fc-element subsets of [N] , and the associated Gaussian random variable is the spectral norm of a random matrix. 
Moreover, in [RV08] the number of rows of the subsampled DFT matrix is a random variable concentrating around 
its expectation, contrary to our case where it is a fixed number. We believe that the former difference in our setup 
may make the proof accessible to a broader audience. 



5 



Organization of the paper. The rest of the paper is organized as follows. After fixing some 
notation, in Section 2 we prove the average-case Johnson bound that relates a lower bound on 
average pair-wise distances of subsets of codewords in a code to list decoding guarantees on the 
code. We also show, in Section 2.3, an application of this bound on proving list decodability of 
"locally sparse" codes, which is of independent interest and simplifies some earlier list decoding 
results. In Section 3, we prove our main theorem on list decodability of random linear codes 
by demonstrating a reduction from RIP-2 guarantees of DFT-based complex matrices to average 
distance of random linear codes, combined with the Johnson bound. Finally, the RIP-2 bounds on 
matrices related to random linear codes are proved in Section 4. 

Notation. Throughout the paper, we will be interested in list decodability of g-ary codes. We 
will denote an alphabet of size q by [q] (which one can identify with {0, 1, . . . , q — 1}); for linear 
codes, the alphabet will be W q , the finite field with q elements (when q is a prime power). 



We use the notation i := \^-T. When / < Cg (resp., / > Cg) for some absolute constant 

C > 0, we use the shorthand / < g (resp., / > g). We use the notation log(-) when the base of 
logarithm is not of significance (e.g., / < logx). Otherwise the base is subscripted as in \og b {x). 
The natural logarithm is denoted by ln(-). 

For a matrix M and a multiset of rows T, define Mt to be the matrix with |T| rows, formed 
by the rows of M picked by T (in some arbitrary order). Each row in Mt may be repeated for the 
appropriate number of times specified by T. 

2 Average-distance based Johnson bound 

In this section, we show how the average pair-wise distances between subsets of codewords in a 
g-ary code translate into list decodability guarantees on the code. 

Recall that the relative Hamming distance between strings x, y £ [q] n , denoted 5(x, y), is defined 
to be the fraction of positions i for which xi^yi. The relative distance of a code C is the minimum 
value of S(x,y) over all pairs of codewords x / y G C. We define list decodability as follows. 

Definition 2. A code C C [q] n is said to be (p, £)-list decodable if Vy G [q] n , the number of 
codewords of C within relative Hamming distance less than p is at most £. A 

The following definition captures a crucial function that allows one to generically pass from 
distance property to list decodability. 

Definition 3 (Johnson radius). For an integer q>2, the Johnson radius function J q : [0, 1 — 1/q] — > 
[0, 1] is defined by 



The well known Johnson bound in coding theory states that a q-aiy code of relative distance 
5 is (J q (5 — S/L), L)-list decodable (see for instance [GS01]). Below we prove a version of this 
bound which does not need every pair of codewords to be far apart but instead works when the 
average distance of a set of codewords is large. The proof of this version of the Johnson bound 
is a simple modification of earlier proofs, but working with this version is a crucial step in our 
near-tight analysis of the list decodability of random linear codes. 

4 We require that the radius is strictly less than p instead of at most p for convenience. 
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Theorem 4 (Average-distance Johnson bound). Let C C [q] n be a q-ary code and L > 2 an integer. 
If the average pairwise relative Hamming distance of every subset of L codewords of C is at least 5, 
then C is (J q (5 — S/L),L — l)-list decodable. 

Thus, if one is interested in a bound for list decoding with list size L, it is enough to consider 
the average pairwise Hamming distance of subsets of L codewords. 



2.1 Geometric encoding of g-ary symbols 

We will give a geometric proof of the above result. For this purpose, we will map vectors in [q] n to 
complex vectors and argue about the inner products of the resulting vectors. 

Definition 5 (Simplex encoding). The simplex encoding maps x G {0, 1, • • • , q — 1} to a vector 
<p(x) G C 9 ^ 1 . The coordinate positions of this vector are indexed by the elements of [q — 1] := 
{1, 2, . . . ,q — 1}. Namely, for every a £ [q — 1], we define ip(x){a) := 0J xa where uj = e 27rl//|3 is the 
primitive gth complex root of unity. 

For complex vectors v = (v i, t>2, . . . , v m ) and w = (wi,W2, • • • , w m ), we define their inner product 
(v, w) = YliLi v i w i ■ From the definition of the simplex encoding, the following immediately follows: 

<*.),*»»={ i; 1 

We can extend the above encoding to map elements of [q] n into C" -1 - 9 " 1 ^ in the natural way by 
applying this encoding to each coordinate separately. From the above inner product formula, it 
follows that for x, y G [q] n we have 

(ip(x),ip(y)) = (q- l)n-qS(x,y)n . (1) 

Similarly, we overload the notation to matrices with entries over [q\. Let M be a matrix in [g] nxAr . 
Then, y?(M) is an n(q — 1) x N complex matrix obtained from M by replacing each entry with its 
simplex encoding, considered as a column complex vector. 

Finally, we extend the encoding to sets of vectors (i.e., codes) as well. For a set C C [q] n , <p(C) 
is defined as a (q — l)n x \C\ matrix with columns indexed by the elements of C, where the column 
corresponding to each c G C is set to be ip(c). 



2.2 Proof of average-distance Johnson bound 

We now prove the Johnson bound based on average distance. 

Proof (of Theorem 4)- Suppose {c±, C2, . . . , cl} Q [q] n are such that their average pairwise relative 
distance is at least 5, i.e., 

E S-(f\ ■ (2) 

l<i<j<L ^ ' 

We will prove that ci, C2, . . . , cl cannot all lie in a Hamming ball of radius less than J q {5 — 5/L)n. 
Since every subset of L codewords of C satisfy (2), this will prove that C is (J q (5 — 5/L), L — l)-list 
decodable. 
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Suppose, for contradiction, that there exists cq 6 [q] n such that 5(cq, < p for i = 1, 2, . . . , L 
and some p < J q (5 — S/L). Recalling the definition of J q (-), note that the assumption about p 
implies 

(i — "^tV > 1 — + ~S"7 • 
V g-iy 9-1 ?-lL 

For i = 1, 2, . . . , L, define the vector V{ = <p(cj) — /3ip(co) €. C™^ 1 ), for some parameter j3 to be 
chosen later. By (1) and the assumptions about Co, c\, . . . , cl, we have (<p(cj), </?(co)) > (q—l)n—qpn, 
and Ei<i< j <L( ( ^(ci),^(ci)) < (f)((9- l)n-qSn). We have 

, L L . L 

\ i=l i=l ' i=l l<«<j<L 

< L(n(g - 1) + p 2 n{q - 1) - 20(n(g - 1) - ?pn)) + 

+ L(L - l)(n(q - 1) - gJn + /3 2 n(g - 1) - 2f3(n(q - 1) - qpn)) 

Picking 13 = 1 — and recalling (3), we see that the above expression is negative, a contradiction. 

□ 



2.3 An application: List decodability of Reed-Muller and locally sparse codes 

Our average-distance Johnson bound implies the following combinatorial result for the list decod- 
ability of codes that have few codewords in a certain vicinity of every codeword. The result allows 
one to translate a bound on the number of codewords in balls centered at codewords to a bound 
on the number of codewords in an arbitrary Hamming ball of smaller radius. An alternate proof 
of the below bound (using a "deletion" technique) was given by Gopalan, Klivans, and Zucker- 
man [GKZ08] where they used it to argue the list decodability of (binary) Reed-Muller codes up to 
their relative distance. A mild strengthening of the deletion lemma was later used in [GGR11] to 
prove combinatorial bounds on the list decodability of tensor products and interleavings of binary 
linear codes. 

Lemma 6. Let q > 2 be an integer and rj G (0, 1 — 1/q]. Suppose C is a q-ary code such that 
for every c G C, there are at most A codewords of relative distance less than rj from c (including c 
itself). Then, for every positive integer L > 2, C is (J q (rj — rj/L), AL — l)-list decodable. 

Note that setting A = 1 above gives the usual Johnson bound for a code of relative distance at 
least 77. 

Proof. We will lower bound the average pairwise relative distance of every subset of AL codewords 
of C, and then apply Theorem 4. 

Let ci, C2, • • • , cal be distinct codewords of C. For i = 1, 2, . . . , AL, the sum of relative distances 
of cj, j / i, from q is at least (AL — A)rj since there are at most A codewords at relative distance 
less than rj from a. Therefore 

V Sic c)> ^.(AL-A)n _ A(L-l) 
(M) ^ °^ c i>- AL(AL-l) ~ AL-l 11 ' 

\ 2 ) l<i<j<AL y 1 
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Setting rj' = ^^rr^ , Theorem 4 implies that C is {J q {rf — jfe),AL — l)-list decodable. But 
i]' — ^j- = rj — rj/L, so the claim follows. □ 

3 Proof of the list decoding result 

In this section, we prove our main result on list decodability of random linear codes. The main 
idea is to use the restricted isometry property (RIP) of complex matrices arising from random 
linear codes for bounding average pairwise distances of subsets of codewords. Combined with the 
average-distance based Johnson bound shown in the previous section, this proves the desired list 
decoding bounds. The RIP-2 condition that we use in this work is defined as follows. 

Definition 7. We say that a complex matrix M G £ mxN satisfies RIP-2 of order k with constant 

5 if, for any /c-sparse vector x G C^, we have 5 

(1-S)\\xf 2 < \\Mxf 2 < (1 + <5)||*|||. 

Generally we think of 5 as a small positive constant, say 5 = 1/2. 

Since we will be working with list decoding radii close to 1 — 1/q, we derive a simplified expression 
for the Johnson bound in this regime; namely, the following: 

Theorem 8. Let C C [q] n be a q-ary code and L > 2 an integer. If the average pairwise relative 
Hamming distance of every subset of L codewords of C is at least (1 — l/q)(l — e), then C is 
((1 - l/q)(l - y/e+ l/L),L- l)-list decodable. 

Proof. The proof is nothing but a simple manipulation of the bound given by Theorem 4. Let 

6 ■= (1 - l/q)(l - e). Theorem 4 implies that C is (J g (5(l - l/L)),L - l)-list decodable. Now, 

J s («(l - l/L)) - " 

q \ v y — j- y x ^' / 

n — 1 / / 1 7\ r, — 1 / / 1~\ 

□ 



In order to prove lower bounds on average distance of random linear codes, we will use the 
simplex encoding of vectors (Definition 5), along with the following simple geometric lemma. 

Lemma 9. Let c±,...,cl G [q] n be q-ary vectors. Then, the average pairwise distance 5 between 
these vectors satisfies 




5:= £ <5( Ci ,c,)/(f) 



2 

L' 2 (q-l)n- £iernV( c i 



2 



qL(L - l)n 



5 We stress that in this work, we crucially use the fact that the definition of RIP that we use is based on the 
Euclidean (I2) norm. 
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Proof. The proof is a simple application of (1). The second norm on the right hand side can be 
expanded as 



Y 

ie[L] 



= Y MciW(cj)) 

= L 2 (q — l)n — 2qn Y^ 5(ci,Cj) 

l<i<j<L 
= L 2 (q-l)n-2qn( L )5, 



and the bound follows. 

Now we are ready to formulate our reduction from RIP-2 to average distance of codes. 



□ 



Lemma 10. Let C C [q] n be a code and suppose ip(C)/y/ (q — l)n satisfies RIP-2 of order L with 
constant 1/2. Then, the average pairwise distance between every L codewords of C is at least 

(l-})(l-2(£lj)- 

Proof. Consider any set S of L codewords, and the real vector x G R' c ' with entries in {0, 1} that 
is exactly supported on the positions indexed by the codewords in S. Obviously, ||x||2 = L. Thus, 
by the definition of RIP-2 (Definition 7), we know that, defining M := ip(C), 



\Mx\\ 2 2 < 3L(q - l)n/2. 



(4) 



Observe that Mx = J2i£[L] 9 9 ( c «)- Let 5 be the average pairwise distance between codewords in S. 
By Lemma 9 we conclude that 



5 = 



L 2 (q - l)n - J2ie[L] <P( C 



29® 



n 



W (L 2 - 1.5L)(q - l)n 
qL(L - l)n 



i 1 2(L-1))' 



□ 



We remark that the exact choice of the RIP constant in the above result is arbitrary, as long as 
it remains an absolute constant. Contrary to applications in compressed sensing, for our application 
it also makes sense to have RIP-2 with constants larger than one, since the proof only requires the 
upper bound in Definition 7. 

By combining Lemma 10 with the simplified Johnson bound of Theorem 8, we obtain the 
following corollary. 

Theorem 11. Let C C [q] n be a code and suppose ip(C)/ \J (q — l)n satisfies RIP-2 of order L with 
constant 1/2. Then C is ((l - |) (l - \Jz^i)' L ~ ^j- list decodable. □ 
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The matrix <p(C) for a linear code C CF™ has a special form. It is straightforward to observe 

that, when q = 2, the matrix is an incomplete Hadamard- Walsh transform matrix with 2 k columns, 
where k is the dimension of the code. In general <p(C) turns out to be related to a Discrete Fourier 
Transform matrix. Specifically, we have the following observation. 

Observation 12. Let C C F™ be an [n,k] linear code with a generator matrix G 6 W kxn , and 
define N := q k . Consider the matrix of linear forms Lin G F^ xJV with rows and columns indexed 
by elements of¥ k and entries defined by 

Lin(x,y) := (x,y), 

where (-, •) is the finite-field inner product over ¥ k . Let T C F^ be the multiset of columns of G. 
Then, <f(C) = (^(Liny) (recall, from Definition 5, that the former simplex encoding is applied to the 
matrix enumerating the codewords of C, while the latter is applied to the entries of a submatrix of 
Linj. 

When G is uniformly random, C becomes a random linear code and ip(C) can be sampled by 
the following process: Arrange n uniformly random rows of Lin, sampled independently and with 
replacement, as rows of a matrix M. Then, replace each entry of M by its simplex encoding, seen 
as a column vector in C 9-1 . The resulting complex matrix is <p(C). 

The RIP- 2 condition for random complex matrices arising from random linear codes is proved 
in Theorem 14 of Section 4. We now combine this theorem with the preceding results of this section 
to prove our main theorem on list decodability of random linear codes. 

Theorem 13 (Main). Let q be a prime power, and let e, 7 > be constant parameters. Then for 
all large enough integers n, a random linear code C C F™ of rate R, for some 



~ log(l/7)log 3 (g/e)log<? 
is ((1 — l/q)(l — e),0(l/e 2 ))-list decodable with probability at least 1 — 7. 

Proof. Let C C F™ be a uniformly random linear code associated to a random Rn x n generator 
matrix G over W q , for a rate parameter R < 1 to be determined later. Consider the random matrix 
M = 93(C) = y(Lin^) from Observation 12, where \T\ = n. Recall that M is a (q— l)n x N complex 
matrix, where iV = q Rn . Let L := 1 + [1.5/e 2 ] = 0(l/e 2 ). By Theorem 14, for large enough N 
(thus, large enough n) and with probability 1 — 7, the matrix Mj \/{q — l)n satisfies RIP-2 of order 
L with constant 1/2, for some choice of \T\ bounded by 

n = \T\ < log(l/ 7 )L log(AT) log 3 (gL). (5) 

Suppose n is large enough and satisfies (5) so that the RIP-2 condition holds. By Theorem 11, this 
ensures that the code C is ((1 — l/q)(l — e), 0(l/e 2 ))-list decodable with probability at least 1 — 7. 

It remains to verify the bound on the rate of C. We observe that, whenever the RIP-2 condition is 
satisfied, G must have rank exactly Rn, since otherwise, there would be distinct vectors x, x' G F^ n 
such that xG = x'G. Thus in that case, the columns of M corresponding to x and x' become 
identical, implying that M cannot satisfy RIP-2 of any nontrivial order. Thus we can assume that 
the rate of C is indeed equal to R. Now we have 

R = log q \C\/n = log N/(n\ogq) 

f log N 

~ log(l/7)^log(iV)log 3 ( ( /L)logg' 
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Substituting L = 6(l/e 2 ) into the above expression yields the desired bound. 



□ 



4 Restricted isometry property of DFT-based matrices 

In this section, we prove RIP- 2 for random incomplete Discrete Fourier Transform matrices. 
Namely, we prove the following theorem. 

Theorem 14. Let T be a random multiset of rows of Lin, where \T\ is fixed and each element of 
T is chosen uniformly at random, and independently with replacement. Then, for every 5, 7 > 0, 
and assuming N > Nq(5, 7), with probability at least 1 — 7 the matrix ipfLmx) / y/ (q — l)\T\ (with 
(q — l)\T\ rows) satisfies RIP-2 of order k with constant 5 for a choice of \T\ satisfying 

\T\< l ^pl k log(N)log 3 (qk). (6) 

The proof extends and closely follows the original proof of Rudelson and Vershynin [RV08]. 
However we modify the proof at a crucial point to obtain a strict improvement over their original 
analysis which is necessary for our list decoding application. We present our improved analysis in 
this section. 

Proof (of Theorem 14 )■ Let M := <^(Linr). Each row of M is indexed by an element of T and some 
a G F* (recall that TCFj, where N = q k ). Denote the row corresponding to t G T and a G F* 
by M t:(X , and moreover, denote the set of fc-sparse unit vectors in € N by B 2 ,N ■ 

In order to show that M/y/(q — 1)|T| satisfies RIP of order k, we need to verify that for any 
x = (xi, . . . ,xn) G B% ,N , 

\T\(q - 1)(1 - S) < \\Mx\\ 2 2 < \T\(q - 1)(1 + 5). (7) 

In light of Proposition 20, without loss of generality we can assume that x is real- valued (since the 
inner product between any pair of columns of M is real- valued) . 

For i G F™, denote the ith column of M by M % . For x = (x\, . . . ,xn) G B^ N , define the random 
variable 

Ax := \\Mxf 2 -\T\(q-l) 

x iXj (M\Mi), 

i,jSsupp(x) 

where the second equality holds since each column of M has £2 norm \f(q — 1)|T| and ||x||2 = 1. 
Thus, the RIP-condition (7) is equivalent to 

A:= sup |A X | <<S|T|(g-l). (8) 

xet3 k 2 - N 

Recall that A is a random variable depending on the randomness in T. The proof of the RIP 
condition involves two steps. First, bounding A in expectation, and second, a tail bound. The first 
step is proved, in detail, in the following lemma. 

Lemma 15. Let 5' > be a real parameter. Then, E[A] < d'\T\(q — 1) for a choice of \T\ bounded 
as follows: 

\T\<klog(N)log 3 (qk)/5' 2 . 
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Proof. We begin by observing that the columns of M are orthogonal in expectation; i.e., for any 
i,j € F™, we have 

This follows from (1) and the fact that the expected relative Hamming distance between the columns 
of Lin corresponding to i and j, when i / j, is exactly 1 — 1/q. It follows that for every x £ 
E[A X ] = 0, namely, the stochastic process {&-x} x&B k,N is centered. 

Recall that we wish to estimate 



£ := E T A 



Et sup 



^^(M t , Q ,x) 2 -|T|( ? -i; 



tGT aeF* 



(9) 



The random variables {M t>a ,x) and (M t ^ a >,x) are independent whenever t / t'. Therefore, we can 
use the standard symmetrization technique on summation of independent random variables in a 
stochastic process (Proposition 21) and conclude from (9) that 



£ < £ 1 : = E T E g sup (M t>a ,x) s 



(10) 



where Q := {gt)t&T is a sequence of independent standard Gaussian random variables. Denote the 
term inside Ey in (10) by £t] namely, 

£ T := Eg sup l^9t E ( M t,a,x)' 

Now we observe that, for any fixed T, the quantity £t defines the supremum of a Gaussian 
process. The Gaussian process {G x } x£B k,N induces a pseudo-metric || • \\x on B^ ,N (and more 

generally, C^), where for x,x' G B*'^ ', the (squared) distance is given by 
\\x-x'f x := E G \G X -G X ,\ 2 

= Ei E< M *^> 2 - E ( M t^ x ')' 

teT \aeF% aeF* 



= E \Y.( M ^ x+x '^ Mt ^ x - x ') 

By Cauchy-Schwarz, (11) can be bounded as 



(11) 



-Ax < E( Y,( M ^ x+x ') 2 ) I J2( M ^ x - x ') 2 \ ( |2 > 



-EE ( M ^ x + x ') 2 max [ Yl ( M t,*,x - x'f 



teT agFi 



,QGF* 



(13) 
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Here is where our analysis differs from [RV08]. When q = 2, (13) is exactly how the Gaussian 
metric is bounded in [RV08]. We obtain our improvement by bounding the metric in a different 
way. Specifically, let 77 G (0, 1] be a positive real parameter to be determined later and let r := 1 + 7/ 
and s := 1 + 1/t/ such that l/r + l/s = l. We assume that r\ is so that s becomes an integer. We 
use Holder's inequality with parameters r and s along with (12) to bound the metric as follows: 



\teT aeF* 



l/2r 



/\2 



(£(£<^-*'> 2 ) s ) 



l/2s 



• (14) 



Since ||x||2 = 1, x is /c-sparse, and |M tiQ | = 1 for all choices of (t,a), Cauchy-Schwarz implies that 
(Mt jQ ,,x) 2 < k and thus, using the triangle inequality, we know that 

^2 {M t ^x + x') 2 <Aqk. 

Therefore, for every t G T, seeing that r = 1 + r], we have 

( ^(M t , a ,x + x') 2 ) r < (4gfc)" ^(M^x + x') 2 , 



which, applied to the bound (14) on the metric, yields 



l/2r 



x — X 



'Ik < (W /2r [^ £<M t , Q ,x + x') 

tGTaeF* 



(e(e<^-*'> 2 ) s ) 



l/2s 



.(15) 



£2 



Now, 



tET Q6F* 



^GTaeF* 



where we have defined 



a: 

Observe that, by the triangle inequality, 



x€B$' N t€T aeF* 



£' T < sup 

xeB k 2 > 

Plugging (17) back in (15), we so far have 



££<M t!Q ,x) 2 -|T|(g-l) 

t&T oeF* 



+ \T\{q~l). 



\x-x'\\ x <A{A q kf^£' T l/2r (^(^(M tiQ ,x-x') 2 ) S ) 



l/2s 



(16) 



(17) 



(18) 



(19) 
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For a real parameter u > 0, define Nx(u) as the minimum number of spheres of radius u 
required to cover B% ,N with respect to the metric || • \\x- We can now apply Dudley's theorem on 
supremum of Gaussian processes (cf. [LT91, Theorem 11.17]) and deduce that 

poo 

£t< / y/\ogN x {u)du. (20) 

Ju=0 

In order to make the metric || • \\ x easier to work with, we define a related metric || • \\x' on 
2?2 ,JV \ according to the right hand side of (19), as follows: 

||x-x'||f,:=^(^(M tiQ ,x-x') 2 ) S . (21) 

Let K denote the diameter of B^ ,N under the metric || • \x' ■ Trivially, K < 2\T\ 1 / 2s \/qk. By (19), 
we know that 

\\x - x'\\x < A{Aqkf ,2r £' T l,2r \\x - x'\\ x >. (22) 

Define Nx'(u) similar to Nx(u), but with respect to the new metric X' . The preceding upper 
bound (22) thus implies that 

N x (u) < N X '(u/(4(4qk)^ 2r £ T 1/2r )). (23) 

Now, using this bound in (20) and after a change of variables, we have 

f'°° 

S T < {4qky/ 2r £' T l/2r / y/\ogN x ,(u)du. (24) 

Ju=0 

Now we take an expectation over T. Note that (18) combined with (9) implies 

W, T £ T < £ + \T\(q- 1). (25) 

Using (20), we get 

(10) 

£ 2r < £ 2r = (E T £ T ) 2r < M T £ T r 



2r 



< {4qk)m T ({£' T ) l ' 2r r yJ\ogN x ,{u)d 

V Ju=0 

< (Aqk)^ (E T £ T )m^ ( [ y/log N x <{u)d 

T \Ju=0 

(25) / r°° l N 2r 

< {4qk) ri (£ + \T\(q- l))max I J y/ log N x >(u)du 



2r 

U 



Define 



P x 1/(1+2??) 

£: ^' 1 ^) ■ (26) 



Therefore, recalling that r = 1 + r], the above inequality simplifies to 

r -K \ 1+1/(1+2*1) 



J _ q y/\og N x >{u)du\ , (27) 
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where we have replaced the upper limit of integration by the diameter of B 2 ' under the metric 
|| • \\x' (obviously, Nx>(u) = 1 for all u > K). 

Now we estimate Nx>(u) in two ways. The first estimate is the simple volumetric estimate 
(cf. [RV08]) that gives 

logiV X /(u) < klog(N/k) + Hog(l + 2K/u). (28) 
This estimate is useful when u is small. For larger values of u, we use a different estimate as follows. 
Claim 16. logiV x ,(u) < \T\ l I s (log N)qks / u 2 . 

Proof. We use the method used in [RV08] (originally attributed to B. Maurey, cf. [Car 85, § 1]) and 
empirically estimate any fixed real vector x = (xi, ... ,xn) £ B 2 ,N by an m-sparse random vector 
Z, for sufficiently large m. The vector Z is an average 

IT. m 

Z--=— £^ ( 29 ) 

where each Zi is a 1-sparse vector in € N and E[Zj] = x/ \f~k~. The Zj are independent and identically 
distributed. 

The way each Zj is sampled is as follows. Let x' := xj\fk so that ||x'||i = < 1. With 
probability 1 — \\x'\\, we set Zi := 0. With the remaining probability, Zi is sampled by picking a 
random j G supp(x) according to the probabilities defined by absolute values of the entries of x', 
and setting Zi = sgn(x^)e-,-, where ej is the jth standard basis vector 6 . This ensures that E[Zj] = x'. 
Thus, by linearity of expectation, it is clear that E[Z] = x. Now, consider 

£ 3 :=E||Z-x|| X '. 

If we pick m large enough to ensure that £3 < u, regardless of the initial choice of x, then we can 
conclude that for every x, there exists a Z of the form (29) that is at distance at most u from x 
(since there is always some fixing of the randomness that attains the expectation). In particular, 
the set of balls centered at all possible realizations of Z would cover B 2 ' N ■ Since the number of 
possible choices of Z of the form (29) is at most (2N + l) m , we have 

logiVx-'H < mlogiV. (30) 

In order to estimate the number of independent samples m, we use symmetrization again to 
estimate the deviation of Z from its expectation x. Namely, since the Zj are independent, by the 
symmetrization technique stated in Proposition 21 we have 

(31) 

X' 

where (ej)jg[ m ] is a sequence of independent Rademacher random variables in {— 1, +1}. Now, 
6 Note that, since we have assumed a; is a real vector, sgn(-) is always well-defined. 



£ 3 <^-E 
m 



E 

i=i 



tiZi 
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consider 



2s 



E 



£4 '■= E ^2 e iZi 
i=i 

E(E<^E^> 2 ) S 

teT aeF* i=l 
/ m 

E E E (E e *< M ^>)' 



ter \aeF* i=l 



= E E E e ^ E (Mt, Q ,^)(M t , Q ,^r 

t£T \i,j=l aeF* 

Since the entries of the matrix M are bounded in magnitude by 1, we have 

< a. 



(32) 



2<M t)Q ,Z i )(M tia ,Z 7 -)* 



Using this bound and Proposition 22, (32) can be simplified as 



£a = E 



E e *^ 



i=i 



2* 



< |T|(4gras) s , 



and combined with (31), and using Jensen's inequality, 

£ 3 < {T^y/^qks/m. 

Therefore, we can ensure that £3 < u, as desired, for some large enough choice of m; specifically, 
for some m < \T\ l l s qks/u 2 . Now from (30), we get 



\ogN x ,{u) < ITI 1 ^ (log N)qks/u 2 . 



Claim 16 is now proved. 



(33) 
□ 



Now we continue the proof of Lemma 15. Break the integration in (27) into two intervals. 
Consider 



(■A t-K 
£ 5 := / y/logN X '(u)du+ / y/\ogN x 

Ju=0 Ju=A 



(u)du, 



£a £7 

where A := K/yfqk. We claim the following bound on £5. 
Claim 17. £ 5 < \T\V 2s ^f(^g~ N)qks log(gfc). 

Proof. First, we use (28) to bound £q as follows. 



( i- A 

£ 6 < A^Jk\og(N/k) + Vk V ln (! + 2K/u)du. 

Ju=0 



(34) 
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Observe that 2K/u > 1, so 1 + 2K/u < AK/u. Thus, 

j y/]n(l + 2K/u) du < / y/\n.(±K/u) du 
Jo Jo 

[-A/2K 

= 2K y/]n(2/u) du 

Jo 

= 2K f A y/M4K/A) + (l - erf (VM^M))) 



A^/MAK/A) + 2^K erfc ( y/]n(4K /A)) , (35) 



where erf(-) is the Gaussian error function erf(x) := f^ =0 e t2 dt, and erfc(x) := 1 — erf(x), and 
we have used the integral identity 

J y / ln{l/x)dx = -^erf (vW*)) + xy/\n(l/x) + C 

that can be verified by taking derivatives of both sides. Let us use the following upper bound 

2 r°° 2 2 r°° t 2 1 e~ x2 
Nx > 0) erfc(ar) = -= / e~ l dt < — -e~* dt = — , 

y/TT J t=x yjTT J t=x X VVT X 

and plug it into (35) to obtain 

I V ln (! + 2K / U ) du < Ay/]n(4K/A) + 2^K ( -L • " ' 

" V V 71 " 



= A^/MAK/A) + 



2^/1x1(4^/^) 



< AVbgM<m 1/2s 0oip)), 

where the last inequality holds since ,4 = if/y^ < ITI 1 / 23 . Therefore, by (34) we get 

£ 6 < \T\ 1 / 2s Vk( v / l^N + V^iP)). (36) 
On the other hand, we use Claim 16 to bound £7. 

v Ju=A 

< iT^WilogNJqkslogiqk). (37) 

Combining (36) and (37), we conclude that for every fixed T, 

8 b = 8 Q + 8 7 < iTl^VOogJVjgfcalogCgfe). 
Claim 17 is now proved. □ 

By combining Claim 17 and (27), we have 

1 + 1/(1+27/) 



< 



(4^)" (|T| x / 2s y/ {log N)qks log(gfc)) 



= (4gfe) , '|rr'/( 1+2, '^V(logiV)gfcslog(gfc)) 1+1/(1+2, ' ) . (38) 
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By Proposition 23 (setting a := £/(\T\(q — 1)) and jjl := 2i]), and recalling the definition (26) of £ , 
in order to ensure that £ < S'(q— 1)\T\, it suffices to have 

2(1+1) 

£ < 8' |T|(g- l)/4. (39) 
Using (38), and after simple manipulations, (39) can be ensured for some 

|T| <^^k(logN)log 2 (qk)/5' 2 . 

This expression is minimized for some r] = 1/B (log (<?&)), which gives 

|T| <A;(log7V)log 3 (^)/<5' 2 . 

This concludes the proof of Lemma 15. □ 

Now we turn to the tail bound on the random variable A and estimate the appropriate size 
of T required to ensure that Pr[A > 5|T|((/ — 1)] < 7. We observe that the tail bound proved in 
[RV08] uses the bound on E[A] as a black box. In particular, the following lemma, for q = 2, is 
implicit in the proof of Theorem 3.9 in [RV08] (the extension to arbitrary alphabet size q requires 
only syntactical modifications to the exact argument in [RV08]). 

Lemma 18. [RV08, implicit] Suppose that, for some 8' > 0, E[A] < 8'\T\(q — 1). Then, there are 
absolute constants c±, 02,03 such that for every A > 0, 

Pr[A>(ci + c 2 A)J'|r|((/-l)] <3exp(-A 2 ), 

provided that 

\T\/k> c 3 V\/5'. (40) 

□ 



Now it suffices to instantiate the above lemma with A := ^^1(3/7) and 8' := 8/{c\ + C2A) = 
8/&( yln(3/7)), and use the resulting value of 8' in Lemma 15. Since Lemma 15 ensures that 
\T\/k = fi(logiV), we can take N large enough (depending on <5, 7) so that (40) is satisfied. This 
completes the proof of Theorem 14. □ 

The proof of Theorem 14 does not use any property of the DFT-based matrix other than 
orthogonality and boundedness of the entries. However, for syntactical reasons, that is, the way 
the matrix is defined for q > 2, we have presented the theorem and its proof for the special case 
of the DFT-based matrices. The proof goes through with no technical changes for any orthogonal 
matrix with bounded entries (as is the case for the original proof of [RV08]). In particular, we 
remark that the following variation of Theorem 14 also holds: 



Theorem 19. Let A € € NxN be any orthonormal matrix with entries bounded by 0(l/y/~N). Let T 
be a random multiset of rows of A, where \T\ is fixed and each element ofT is chosen uniformly at 
random, and independently with replacement. Then, for every 8, 7 > 0, and assuming N > Nq(8, 7), 
with probability at least 1 — 7 the matrix (^J N /\T\)At satisfies RIP-2 of order k with constant 8 
for a choice of \T\ satisfying 

|T| < 10g( ^ 2 /7) fc(logJV)log 3 fc. □ 
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A Useful tools 



The original definition of RIP-2 given in Definition 7 considers all complex vectors x € C n . Below 
we show that it suffices to satisfy the property only for real- valued vectors x. 

Proposition 20. Let M € <E mxN be a complex matrix such that M^M G ~R NxN and for any 
k-sparse vector x 6 R^, we have 

(1 -(y)Wl < \\Mxg < (1 + S)\\x\\ 2 2 . 

Then, M satisfies RIP-2 of order k with constant 5. 

Proof. Let x = a + ib, for some a, b 6 H N , be any complex vector. We have \\x\\2 
and 



MI2 + 



II, 



\Mxf 2 



MI2 



x^M^Mx - \\x\\l 



(*) 



(**) 



(a f - i6 t )M t M(a + i6) - ||x| 
a)M^Ma + b^M^Mb + i(a t M t M6 - b+M+Ma) - \\x\ 
a)M^Ma + b ] M^Mb- \\x\ 



a)M^Ma - \\a\\l + b^M^Mb 



< 5\\a\\i + 6\\b\\i 
= S\\x\\l 

where (*) is due to the assumption that M^M is real, which implies that a^M^Mb and b^M^Ma 
are conjugate real numbers (and thus, equal), and (**) is from the assumption that the RIP-2 
condition is satisfied by M for real-valued vectors and the triangle inequality. □ 

As a technical tool, we use the standard symmetrization technique summarized in the follow- 
ing proposition for bounding deviation of summation of independent random variables from the 
expectation. The proof is a simple convexity argument (see, e.g., [LT91, Lemma 6.3] and [Verl2, 
Lemma 5.70]). 

Proposition 21. Let (-Xj)j e [ m ] be a finite sequence of independent random variables in a Banach 
space, and (ej)j e r m i and (gi)i^\ m ] be sequences of independent Rademacher (i.e., each uniformly 
random in {— and standard Gaussian random variables, respectively. Then, 

Ell (*i ~ HXi}) < Ell e ^ £ E || E 9i*i 



ie m 



ie m 



ie\m\ 



More generally, for a stochastic process (X, 



)id\m\ ,t 



e j- where T is an index set, 



E sup II ^ {X\ T) - nx\ T) ]) < E sup || ^ ei XP < E sup II ^ 9i x\ 



'(r) 



(r) 



At) 



ie m 



ie\m\ 



rST 1 



ie m 



□ 
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The following bound is used in the proof of Claim 16, a part of the proof of Lemma 15. 

Proposition 22. Let (ej)j e [ m j be a sequence of independent Rademacher random variables, and 
( a ij)i,je[m] be a sequence of complex coefficients with magnitude bounded by K. Then, 



Ef V 



i,je[m] 



< (AKms) 



Proof. By linearity of expectation, we can expand the moment as follows. 

i,je[m] 



.)*- E 



(ii,...i s )e[m] i 

(iiv.j.JeH 1 



)■ 



Observe that Efe^ • • • e% a ej x ■ ■ ■ ej s ] is equal to 1 whenever all integers in the sequence 

(ii,...,i s Ji,...,j s ) 

appear an even number of times. Otherwise the expectation is zero. Denote by S C [m] 2s the set 
of sequences . . . , i s ,ji, ■ ■ ■ ,js) that make the expectation non-zero. Then, 



E 



/ j 



OijCiCj 



(il,...i s ,ji,...j s )eS 



< K S \S\. 



One way to generate a sequence a £ S is as follows. Pick s coordinate positions of a out of the 2s 
available positions, fill out each position by an integer in [m], duplicate each integer at an available 
unpicked slot (in some fixed order) , and finally permute the s positions of a that were not originally 
picked. Obviously, this procedure can generate every sequence in S (although some sequences may 
be generated in many ways). The number of combinations that the combinatorial procedure can 
produce is bounded by ( 2 s s )m s (s!) < (4ms) s . Therefore, \S\ < (4ms) s and the bound follows. □ 

We have used the following technical statement in the proof of Lemma 15. 
Proposition 23. Suppose for real numbers a > 0, fi G [0, 1], S € (0, 1], we have 



(—) 

\l + aJ 



< 



Then, a < 5. 



2+m 1 2+ai 

Proof. Let 5' := <5 1 +m/4i+m > /4. From the assumption, we have 



(—) 
\l + aJ 



i 



<5' 



< <5 2+ ^(l + a)/4. 



(41) 



Consider the function 

/(a) := a 2+ ^ - 5 2+tx a/A - S 2+ "/4. 

The proof is complete if we show that, for every a > 0, the assumption f(a) < implies a < 5; or 
equivalently, a > 5 => f(a) > 0. Note that /(0) < 0, and f"(a) > for all a > 0. The function 
/ attains a negative value at zero and is convex at all points a > 0. Therefore, it suffices to show 
that f(S) > 0. Now, 



f(S) = 5 2+ » - 5 3+ 74 - 6 2+tx /4 > (35 2+fl - 5 3+ ^)/4. 
Since S < 1, the last expression is positive, and the claim follows. 



□ 
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